SHARING OF STILL IMAGES WITHIN 
A VIDEO TELEPHONY CALL 



CROSS REFERENCE TO RELATED APPLICATIONS 

5 

The present application is a continuation-in-part of 
pending U.S. application serial number 10/033,813, filed 
December 20, 2001, entitled "Telephonic Addressing For 
Establishing Simultaneous Voice and Computer Network 

10 Connections", which is a continuation-in-part of prior U.S. 
application serial number 09/978,616, filed October 16, 
2001, entitled "Video Telephony". This application is 
further related to co-pending U.S. applications serial 
number (docket 1794), entitled "Private Sharing of Computer 

15 Resources Over an Internetwork"; and (docket 17 98), entitled 
"Sharing of Prerecorded Motion Video Over an Internetwork," 
both incorporated herein by reference. , 

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH 

20 

Not Applicable. 

BACKGROUND OF THE INVENTION 

25 FIELD OF THE INVENTION 

The present invention relates to establishing a video 
telephony session between users connected to a computer 
network in order to share still images privately between the 

30 users simultaneously with the video telephony call. The 

still images may typically comprise graphics or pictures to 
be shared with family or friends over the Internet. For 
example, the pictures can have been taken using a digital 
still camera or could be scanned photographs. 

35 Internetworking (i.e., the interconnection of many 

computer networks) allows the interaction of very large 
numbers of computers and computer users. The most well 
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known example is the Internet. Computers connected to the 
Internet may be widely separated geographically and utilize 
many different hardware and software configurations. In 
order to achieve communication sessions between any two 
5 endpoints on the Internet, an addressing system and various 
standard protocols for exchanging computer data packets have 
been developed. 

DESCRIPTION OF THE RELATED ART 

10 

Each packet sent over the Internet includes fields 
that specify the source and destination address of the 
packet according to Internet Protocol (IP) addresses 
assigned to the network interface nodes involved. Currently 

15 assigned addresses comprise 32 bits, although future 

standards allow for 128 bit addresses. The 32 bit addresses 
are normally written by breaking the 32 bits into 4 groups 
of 8 bits each and writing the decimal equivalents of each 
group separated by periods (e.g., 208.25.106.10). 

20 Since numerical IP addresses are inconvenient to use 

and remember, a protocol for assigning and accessing logical 
names is used known as the domain name system (DNS) . DNS 
servers are deployed within the Internet which perform a 
translation function between a logical domain name such as 

25 "sprint.com" and its numerical equivalent "208.25.106.10". 
After receiving an IP address back from a DNS server, a 
computer can forward data packets to the IP address and 
establish a connection or session with the remote computer. 

While the DNS system works well for hosted content 

30 (e.g., material made available for browsing by commercial 
and private entities) , it is not well suited to ad hoc 
communications or exchanges of data between individuals. 
Hosting a website and registering an IP address within the 
DNS system is expensive and time consuming. Furthermore, 

35 due to an impending shortage of IP addresses and the cost 
for maintaining use of each IP address, many Internet 



service providers assign IP addresses dynamically to their 
individual users. In other words, when a user signs on to 
their service, they are temporarily assigned an IP address 
from an address pool assigned to their service provider. 
5 The user occupies that IP address only for their current 
session. 

Even when individual users have their own static IP 
addresses, and when other users can remember the IP address 
of a user with whom they would like to establish a 

10 connection session over the Internet (e.g., for voice or 
video telephony) , the need to configure their hardware or 
software is too complex for many users. This is one reason 
why e-mail is such a popular and successful Internet 
application. A mail server with an easy to remember domain 

15 name acts as intermediary between two individual users. 
Using a simple application program and the recipient's 
account name on the mail server (i.e., their e-mail 
address) , text messages and computer files can be exchanged. 
The exchange, however, does not allow the users to interact 

20 in real time. Thus, there is a need for a way to allow two 
or more individual users to establish interactive connection 
sessions over the Internet without requiring overt knowledge 
of the other's IP address and without complicated 
configurations or set-ups. 

25 Copending applications U.S. Serial No. 09/978,616 and 

U.S. Serial No. 10/033,813 teach the use of a central server 
allowing two or more individual users to establish 
interactive connection sessions over the Internet without 
requiring overt knowledge of the other's IP address and 

30 without complicated configurations or set-ups. Each user 
registers with the central server, resulting in a database 
of users and their current IP addresses. A calling user 
sends a request to the central server to establish a 
connection with a called user. The central server can 

35 either relay all network message packets between the users 
for the duration of a "call", or it may provide the IP 



addresses to the users so that they can exchange packets 
directly. The called user may be identified within the 
database by information well known or easily discovered by 
other users, such as their telephone number. A telephone 

5 call may be established simultaneously with establishing the 
computer network session, thereby enhancing the user 
interaction regardless of the type of computer data to be 
exchanged (e.g., video frames, computer files, etc.). In 
one embodiment, the computer network session is 

10 automatically established in response to the act of dialing 
the called user's telephone number. 

The functions of identifying the called telephone 
number, forwarding a call request to the central server, and 
conducting a packet exchange during a data call are 

15 performed by a specific software application program 

referred to herein as a call client. A particular call 
client may include provision for exchanging certain types of 
data for preselected purposes and according to predefined 
protocols. In particular, the call client handles the 

20 transmission of live video images from a video camera 

coupled to the computer and the reception and displaying of 
live video images sent from the other user. 

During a video telephony call, it would be desirable 
for the users to share additional types of data, such as 

25 still images or photographs, without requiring complicated 
set-up or installation or complex procedures. 

SUMMARY OF THE INVENTION 

30 The present invention provides the ability to share 

still images simultaneously to both users in an easy to use 
system. 

In one aspect of the invention, a method is provided 
for sharing still images between first and second computers 
35 connected to an internetwork for exchanging network packets 
therebetween, wherein each of the computers has a respective 
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private IP address within the internetwork. A central 
server coupled to the internetwork contains a database of IP 
addresses of registered computers. First and second call 
clients run in the first and second computers, respectively, 

5 for establishing a data call between the first and second 
computers in response to the database of IP addresses. The 
data call comprises live video exchange from at least one 
video camera coupled to one of the first and second 
computers and comprises a network session between the first 

10 and second call clients. A voice telephone call is 

established between first and second users of the first and 
second computers, respectively, and at least one of the 
first and second users is seen in the live video exchange. 
The first user initiates or launches a first image viewer 

15 subclient under control of the first call client. The first 
image viewer subclient loads and displays still image data 
specified by the first user on the first computer. The 
first image viewer subclient transmits the still image data 
to the second computer using the network session. The 

20 second call client in the second computer receives the still 
image data, runs a second image viewer subclient, and loads 
the still image data into the second image viewer subclient. 
The second image viewer subclient displays the still image 
data on the second computer. The live video exchange is 

25 maintained simultaneously with display of the still image 
data by the image viewer subclients. 

BRIEF DESCRIPTION OF THE DRAWINGS 

30 Figure 1 is a block diagram showing the 

interconnection of users over the Internet to the central 

server of the present invention. 

Figure 2 is a block diagram showing a user connection 

model of the present invention. 
35 Figure 3 is a flow diagram of the establishment of a 

data call used in the present invention. 
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Figure 4 is a block diagram showing a first 
embodiment of packet flow for a data call. 

Figure 5 is a block diagram showing a second 
embodiment of packet flow for a data call. 
5 Figure 6 is a block diagram showing the elements 

within each computer for accomplishing the sharing of still 
images between the computers. 

Figure 7 is a block diagram showing the elements of 
the computers in greater detail. 
10 Figure 8 is a flowchart showing a preferred 

embodiment of a method for sharing still images in 
conjunction with a video telephony call. 

Figure 9 is a block diagram graphically depicting the 
operation of the present invention. 

15 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

Referring to Figure 1, a plurality of user computers 
10, 11, and 12, and a central server 13 are internetworked 

20 via the Internet 14. A plurality of routers 15 within 
Internet 14 direct packets between various endpoints or 
nodes. Computers 10 and 11 are shown as being connected to 
Internet routers belonging to Internet Service Providers 
(ISP's) 16 and 17, respectively. The connections to the 

25 ISP's may be by dial-up, digital subscriber line (DSL), 

cable modem, or integrated access device (IAD), for example. 
Central server 13 and computer 12 are shown directly 
connected to a router. 

Network communication comprises data messages or 

30 packets transferred between separate endpoints, such as 
between computers 10, 11, or 12 (as clients) and central 
server 13. The packet transfer is accomplished by routers 
15 using the IP addresses contained in each packet. Central 
server 13 typically has a fixed IP address that is listed on 

35 the DNS servers accessible to each computer. Each computer 
user can easily communicate with central server 13 by 
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supplying its logical name (e.g., www.sprint.exchange.com) 
which is automatically resolved by their browser into an IP 
address by consulting a DNS server. Exchanging packets 
between users 10, 11, and 12 themselves cannot usually be 

5 accomplished in the same way because the users and their IP 
addresses are not listed in the DNS system. Furthermore, 
users 10, 11, and 12 may not wish to allow remote access 
into their computers except in certain circumstances. 

The present invention facilitates exchanging data 

10 messages between two individual users by providing a 

specialized directory or look-up within central sever 13. 
As shown in Figure 2, the present invention may be used 
within a system that functions to simultaneously establish a 
voice telephone call between the two individual computer 

15 users. In certain embodiments, the voice call serves as the 
user action that initiates the computer processing to 
establish the computer-to-computer connection. In addition, 
the voice call provides a way to alert the called party of 
the request to establish the computer connection and then 

20 serves to enhance the interaction between the two users 

during the exchange of computer data. However, the present 
invention also provides other methods for initiating the 
computer processing, and a simultaneous voice telephone call 
is not necessary in the present invention. 

25 Regarding the embodiment with a simultaneous voice 

telephone call in Figure 2, computers 10 and 11 have 
associated telephones 18 and 19 used by the same respective 
users. The computers and telephones may be fixed 
installations (e.g., in a residence or a business office) or 

30 may be mobile devices (e.g., laptop computer and cellular 
phone) , as long as both are accessible to each user at the 
same time. The telephones are connected to the public 
switched telephone network (PSTN) 20. Central server 13 
provides a user look-up and interconnecting service for 

35 registered users. For security and/or billing purposes, 
access to the service preferably is tied to user ID's and 



passwords. A user may be given an ID and password with 
initial sign-up for the service. Each user would manually 
configure the telephone number that they want to be 
associated with. When the user is "on-line" (i.e., has 

5 their computer turned on and connected to Internet 14), 
their computer sends a registration message to central 
server 13 to notify it that the user is available. Central 
server 13 can inspect the registration message to determine 
the current IP address and port number at which the user 

10 resides for its current connection session. Alternatively, 
the user may manually configure their IP address in some 
circumstances. Upon registration, central server 13 may 
preferably determine whether the user has a respective 
firewall as described in copending U.S. application serial 

15 no. 10/034,012, incorporated herein by reference. In any 
case, central server 13 contains a database of currently 
active, registered users. Each user entry in the database 
includes fields for user ID, password, telephone number, and 
IP address (including port number), user status, and a 

20 firewall flag, for example. 

In the connection model of Figure 2, a user #1 dials 
telephone 18 to make a voice call to a user #2 at telephone 
19. The telephone number dialed by user #1 is captured as a 
target telephone identifier number and sent to a call client 

25 (i.e., an application program for managing the data call) in 
computer 10 being used by user #1. Computer 10 forwards the 
target telephone number to central server 13 as part of an 
access request for establishing a connection with user #2. 
Central server 13 looks up the target number in its 

30 database. When it finds the target number, central server 
13 identifies the IP address associated with user #2 and 
sends an initiation message to computer 11 being used by 
user #2. When computer 11 receives the initiation message, 
it launches its own call client. The initiation message may 

35 identify user #1 (preferably by both telephone number and 
user ID) and the type of data to be exchanged (i.e., the 



application program or how the call client should be 
configured to receive the data) . User #2 answers the 
telephone voice call and learns that an initiation message 
was sent to their computer. Using computer 11, user #2 can 

5 verify the calling party as user #1 and can indicate whether 
they accept the computer network connection (i.e., the data 
call) with user #1. Once user #2 accepts, data messages can 
be exchanged between the call client application programs 
running on computers 10 and 11. The call client application 

10 programs can be written to perform file transfers of various 
types of files, video data or frames for video telephony, or 
other real-time data or control signals. It may also be 
desired by a user to share computer data or other computer 
resources besides the data or file types that have been 

15 programmed into the call clients, as described below. 

The sequence of events occurring to establish a data 
call is shown in greater detail in Figure 3, in which user 
#1 events are in the left column, central server events in 
the center column, and user #2 events in the right column. 

20 In step 21, user #1 invokes the real-time interconnection 

service of the present invention. This can be configured as 
part of the normal start-up of their computer or can result 
from manually launching a software application such as the 
call client after start-up has finished. When the service 

25 is invoked by user #1, a registration message is sent to the 
central server in step 22. The registration message 
preferably includes the user ID and password assigned to 
user #1. In addition, the application software that creates 
the registration message may also determine the local IP 

30 address being used by the computer and includes this as data 
within the registration message. The registration message 
would typically also include the telephone number being used 
by user #1. In addition to the IP address being explicitly 
added to the message by the application program for user #1, 

35 the IP address (and port number) is typically embedded in 
each packet forwarded by the network and the central server 



preferably extracts the automatically embedded IP address 
and port number. In step 23, the central server receives 
the registration message and adds the new user to the 
database or updates the user status, as necessary. 

5 Separately, user #2 invokes the real-time 

interconnection service in step 24. User #2 sends a 
registration message in step 25, and the central server 
receives the registration message and adds user #2 to the 
database or updates the user status, as necessary. 

10 Thereafter, the central server may periodically exchange 

further messages with each registered user to keep the user 
status current and to maintain an open session with each 
user, for example. When a user shuts down their application 
program or their computer, an unregister message (not shown) 

15 may also be sent to the central server. 

During the time that user #1 is on-line, user #1 
desires to exchange computer data with user #2. In step 27, 
user #1 initiates an attempt to contact user #2 and set up 
the data exchange. In a preferred embodiment, user #1 may 

20 identify user #2 by virtue of user #2's telephone number. 

This target telephone number may preferably be captured from 
the act of dialing it on user #l's telephone equipment. 
According to one example, a dedicated module may be 
connected to user #l's telephone to detect the DTMF tones 

25 while dialing and to send the dialed number to user #l's 
computer. The target telephone number for user #2 is 
included in an access request message sent to the central 
server in step 28. 

In step 30, the central server looks up the target 

30 telephone number and gets the IP address (and port number) 
associated with user #2. The initiation message is sent by 
the central server in step 31. 

User #2 receives the initiation message in step 32. 
If not already running, the user #2 computer launches the 

35 appropriate call client application for responding to the 

initiation message and then prompts user #2 to either accept 
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or reject the access request. If rejected, then user #2 
generates a reject message in step 33 and sends it to the 
central server. In step 34, the central server forwards the 
reject message to user #1, which then terminates the data 

5 portion of the attempted communication session in step 35 
(the voice telephone call is accepted, rejected, or 
terminated separately) . 

If user #2 accepts the attempted contact and the 
request for data exchange, then user #2 causes their 

10 computer to generate an accept message in step 36 (e.g., by 
clicking an "accept" button in an application interface of 
the call client) and sends it to the central server. In 
step 37, the central server determines any needed 
configurations for accomplishing the data exchange and then 

15 configures the user #1 and user #2 endpoints in step 38. 
The two main configurations for the data exchange will be 
described in connection with Figures 4 and 5 and are 
selected on the basis of detected firewalls, for example. 
The user #1 and user #2 computers accept the configuration 

20 and then begin to exchange the data messages or packets in 
step 39. Other configuration issues, such as the 
configuration of the client application programs exchanging 
the actual data messages can be handled within the access 
request message, the initiation message, the accept message, 

25 and/or other packets exchanged between the endpoints, for 
example . 

A first packet exchange configuration is shown in 
Figure 4 wherein central server 13 performs a relay function 
such that all packets exchanged between computer 10 and 

30 computer 11 pass through central server 13. In other words, 
after a desired user (called party) accepts the data call 
and central server notifies the first user (calling party) 
of the acceptance, both endpoints continue to address their 
sent packets to central server 13. At central server 13, 

35 each packet is redirected by substitution of IP addresses. 
For example, a packet sent from computer 10 including its 
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own IP address as the source address of the packet and the 
IP address of central server 13 as the destination address 
of the packet is modified after being received by central 
server 13 to have the central server's address as its source 

5 address and to have the IP address of computer 11 as its 

destination address. After modification, central server 13 
sends the packet back to its router and on to computer 11. 
The same operations are used to send packets from computer 
11 to computer 10. The embodiment of Figure 4 has the 

10 advantage that greater privacy of a user's IP address is 

maintained since each user's computer only needs to see the 
IP address of central server 13. Furthermore, this 
configuration can readily function in the presence of 
network address translation (NAT) firewalls at the 

15 endpoints. 

Figure 5 shows an alternative configuration in which 
direct packet exchange between computers 10 and 11 is 
realized. Central server 13 provides a look-up function and 
a connection initiation function. If desired user #2 

20 (called party) accepts a data call, then central server 13 
provides the IP address of computer 11 to computer 10 and 
provides the IP address of computer 10 to computer 11. 
Thereafter, each computer can send packets addressed to the 
other computer and the packets are no longer relayed through 

25 central server 13. This embodiment has the advantage that 
central server 13 may be reduced in size since less traffic 
flows through it. 

The use of either connection method of Figure 4 or 
Figure 5 is transparent to the users. Once either type of 

30 data call is established and the call clients are exchanging 
data messages over the internetwork, a video telephony call 
is conducted by exchanging live video images between the 
users . 

Once a video telephony call is established, still 
35 images can be exchanged as shown in Figure 6. Computer 10 
includes a network interface 40 and a call client 41 
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performing the functions already described. A video camera 
45 provides live video images to call client 41 which 
formats video frames for transmission as the video portion 
of the video telephony call. Computer 10 also runs an image 

5 viewer subclient application 42 for loading, displaying and 
transmitting graphical still images (e.g., compressed 
digital photographs) from a still image data memory 43. The 
image data is preferably stored in compressed graphic files, 
such as jpg files. Still images stored in memory 43 may be 

10 obtained from an image source 44 (e.g., a digital camera or 
an optical scanner) connected to computer 10 or could be 
downloaded from other computer sources (e.g., from the 
Internet or from floppy discs). A user interface 46 may, 
for example, include operating system software and 

15 input/output devices (e.g., monitor, mouse, and keyboard) by 
which a user interacts with (e.g., provides user commands 
to) call client 41 and image viewer subclient 42. 

Viewer subclient 42 operates under control of call 
client 41. Call client 41 preferably includes a command for 

20 launching viewer subclient 42 such as a mouse button or a 

pulldown menu for indicating that the user wants to display 
and transmit still images in conjunction with an ongoing 
video telephony call. When it is running, viewer subclient 
42 is linked to call client 41. The image data to be 

25 transmitted from viewer subclient 42 is preferably handled 
using the same IP address and port as are assigned to call 
client 41. Due to the coordinated interaction of call 
client 41 and viewer subclient 42, no separate network 
session needs to be created in order to exchange still 

30 images or subclient control commands with another user. 

Computer 11 includes a network interface 50, a call 
client 51, an image viewer subclient 52, a video camera 53, 
and a user interface 53. Computer 11 may also have local 
still image data accessible by viewer subclient 52, but need 

35 not have any in order to receive and display the transmitted 
still image data from computer 10. 



Figure 7 shows the operation of call client 41 and 
image viewer subclient 42 in greater detail. In 
establishing the data call (e.g., a video telephony call), 
call client 41 creates a network session 47 between itself 
(as referenced within computer 10 by the local IP address of 
computer 10 and the port address used by call client 41) 
and, depending upon the connection mode, either central 
server 13 or remote computer 11 (as referenced within 
computer 10 by a remote IP address and port address which 
were provided by central server 13) . Using conventional 
network protocols, data is exchanged between computers 10 
and 11. One-way or two-way video data is passed between 
session 47 and video software 48. Video software 48 
processes video from the video camera and forwards it to 
session 47. Video software 48 also processes remote video 
data received from session 47 and feeds it to a display 
interface within the overall user interface. 

Prior to viewer subclient 42 becoming active, all 
network traffic through session 47 is routed to/from video 
software 48. Once viewer subclient is active and 
transmitting still images, a switch 49 is activated in call 
client 41 for properly directing the received network 
packets to the correct application. When subclient 42 is 
the one sending still images to a remote user, the image 
data itself is coupled directly to session 47, bypassing 
switch 49. Even while sending, subclient 42 may receive 
network traffic from the remote viewer subclient since 
either subclient can control the still image display (e.g. 
by generating pause, rewind, and other picture browsing 
commands) . These received commands also pass through switch 
49. The switching is preferably based upon a flag or other 
identifying data encoded at the appropriate protocol level 
within the packets generated by either viewer subclient. 

An overall method of the present invention is shown 
in Figure 8. In step 60, multiple users sign-on or register 
with the central server. A calling user launches their call 
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client on their computer in step 61. Preferably, the 
calling user makes a telephone call to the called user, and 
the act of dialing the telephone number may send a signal to 
the computer for automatically launching the call client if 

5 it is not already running. Alternatively, no telephone call 
is necessary and the calling user may enter a telephone 
number or other identifying information of the called user 
into the call client. In step 62, the phone number or other 
identifying information is sent to the central server and a 

10 data call is established with the called user. 

In step 63, a first user (i.e., either the calling or 
called user) initiates their image viewer subclient. The 
first user selects one or more images that they would like 
to transmit to the other user. For example, a series of 

15 photographs may be arranged into an ordered array or 

slideshow. Alternatively, such a slideshow can be defined 
in advance of the video telephony call and then selected in 
step 63. Any parameters for displaying and transmitting 
(i.e., playing back) the array or slideshow are selected by 

20 the user, such as display time for automatic advancing of 
the pictures. 

In step 64, the first user generates a command in the 
user interface for initiating the actual transmission of the 
selected still image data to the other user (e.g., by 

25 selecting a send or start button in the viewer subclient) . 
Consequently, the still image data is transmitted to the 
other user within the existing network session of the video 
telephony call. In step 65, the receiving user's call 
client recognizes the reception of still image data packets 

30 and launches its own image viewer subclient and loads and 
displays the still images as they are received. Thus, the 
image viewer subclients show the same still image or picture 
simultaneously, allowing the two users to view the still 
image and to still see and hear each other at the same time. 

35 During the still image presentation, the call client 

at the receiving end switches incoming network packets 
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between the live video software and the image viewer 
subclient in response to identifying data in the packets. 
Both users watch and control the picture array or slideshow 
in step 67. At the end of the presentation of still images, 
5 the users may terminate their image viewer subclients in 
step 68 . 

The user experience of simultaneous video telephony 
and sharing of still images is shown in Figure 9. Still 
image data as used herein refers primarily to any digitized 

10 still images or graphics in a computer file format 

compatible with the image viewer subclients. Such images 
may typically be generated by a digital still camera 70, for 
example. Images are downloaded from camera 7 0 into computer 
files stored in computer 10 via a universal serial bus (USB) 

15 interface, for example. Computer 10 preferably compresses 
the image data to facilitate transfer over Internet 14 to 
computer 11. 

Computer 10 includes a display monitor 72 and 
computer 11 includes a display monitor 75. During a video 

20 telephony call, call windows 7 3 and 7 6 show live video 

received from the other endpoint of the video telephony data 
call. To share still images, computers 10 and 11 launch 
viewer windows 74 and 77 on monitors 72 and 75, 
respectively, so that both users are seeing the same still 

25 images at the same time. Due to the low bandwidth required 
to send still image data, the video telephony call can be 
easily maintained at the same time thereby allowing the 
users to see each other and to discuss the still images as 
they are viewed. During the still image presentation, 

30 several viewer control are preferably active so that viewing 
of the images is jointly controlled (e.g., either user can 
navigate to a next or previous image or access a menu to 
modify the automatic display parameters by mouse clicking on 
the corresponding control buttons in viewer windows 74 and 

35 77) . Alternatively, the viewer controls may be set up so 
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that only one user (e.g., the sending user) can control the 
viewing of the images. 

While the present invention has been described with 
respect to two users sharing still images, the- invention 
5 also contemplates that three or more users could 

simultaneously view images or participate in a video 
telephony call. In that case, the sending subclient would 
multicast to each of the remote computers, for example. 
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